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SUMMARY 


ACT Academy was launched in March 2018 as a free online resource to 
help students improve their academic skills and prepare for the ACT test. 
This study provides an early look at ACT Academy usage and its effects on 
ACT test scores. 


We found that moderate usage (e.g., 6 hours) of ACT Academy’s quizzes 
and practice tests leads to an improvement in ACT test scores of 1-2 points. 
However, most students have not spent enough time on the platform to 
reap the benefits. 


We also found that the number of ACT Academy users has increased 
steadily over time, and that female, Asian, and higher-achieving students 
are somewhat more likely to use ACT Academy. 


SO WHAT? 


The results suggest that ACT Academy can be an effective and efficient 
tool to improve academic skills important for college and careers. ACT 
Academy can have a greater impact on college readiness if more students 
are aware of it and invest the time needed to benefit from it. 


NOW WHAT? 


The study only examined usage of assessments (quizzes and practice 
tests) within ACT Academy, and did not analyze how item responses or 
usage of the online learning resources related to ACT test score gains. 
Additional research is needed to understand what usage patterns result in 
the greatest gains. 
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Introduction 


What is ACT Academy: A Learning Platform, Test Preparation 
Resource, or Both? 


ACT® Academy™ was launched on March 22, 2018 as a free online resource to help students improve 
their academic skills and prepare for the ACT® test. It provides individualized learning plans to help 
students improve their skills in four subject areas (English, reading, math, and science) tested on the 
ACT. However, it is also possible for students and teachers to incorporate the learning resources 
available in ACT Academy into instruction in the four subject areas more broadly. Therefore, it might be 
useful to consider the similarities and differences between conventional notions of learning and test prep. 


Learning can be broadly defined as “the process of acquiring new or modifying existing knowledge, 
behaviors, skills, values, or preferences and may involve synthesizing different types of information” 
(Schacter, Gilbert, Wegner, & Hood, 2011). Students may acquire new knowledge or modify existing 
knowledge when using ACT Academy to prepare for the ACT. Yet, for many educators, particularly 
classroom teachers, test preparation is not what comes to mind when they think of “learning” or a 
“learning platform.” Instead, test preparation often assumes that students have already learned or at least 
encountered concepts previously; therefore, the goal of most test prep is to review content and shore up 
understanding of concepts as opposed to teaching them from scratch. In addition to reviewing content 
likely to be covered on a test, test prep entails practicing skills needed to demonstrate knowledge in the 
format of the test (Crocker, 2006). 


While ACT Academy may primarily serve a test prep role, it can also be regarded more broadly as a 
learning tool. Students may encounter entirely new concepts when using ACT Academy or may receive 
new instruction on previously introduced concepts. If ACT Academy is an effective learning tool, we would 
expect it to not only result in higher ACT test scores, but for the learning gains to also transfer to other 
external outcomes (e.g., performance in subsequent high school and college courses, performance on 
other assessments) and skills (e.g., critical thinking, reasoning). Alternatively, if ACT Academy is only 
effective as a test prep tool, we would not necessarily expect positive effects on external factors. As part 
of this study, we examine whether use of ACT Academy leads to higher ACT score gains. Future studies 
will examine the effect of ACT Academy usage on other outcomes, such as classroom performance, to 
determine the extent to which learning gains transfer beyond the ACT. 


Features of ACT Academy 


When users arrive at the ACT Academy website, they have either the option of anonymously searching 
and viewing resources accessible via the Resource Library, or they can sign in/register to access the 
quizzes, practice tests, and other features of ACT Academy. There are four primary sections of the ACT 
Academy interface displayed at the top of the screen: Dashboard, Resources, Progress, and Tips & 
Strategies. A fifth area of ACT Academy, Assignments, is available for students who are members of an 
ACT Academy class set up by a teacher. 
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Dashboard 


The Dashboard provides users an overview of the 
progress made towards mastery in each of the four 
subject areas by displaying one to three green stars 
and indicating the number of quizzes completed in 
each area. On the Dashboard screen, users can 
scroll down to access a daily study plan with 
selected learning resources and a quiz to complete 
each day. Students who have already taken the 
PreACT® or ACT have the option of entering their 
reporting category scores, which indicate how a 
student performed on specific skill areas within each 
of the four subject areas. The reporting categories 
correspond to the domains used to group the ACT 
College and Career Readiness Standards.’ Math contains seven reporting categories, while science, 
English, and reading each contain three reporting categories. Examples include Number & Quantity for 
math and Production of Writing for English. 


Resources 


The Resources section presents students with a selection of learning resources organized by topic under 
each of the four subject areas. ACT Academy includes both free resources (Open Educational 
Resources, or OERs) and links to premium resources students can purchase. Premium resources are 
mostly homework assignments, while the OERs consist primarily of video-based, mini-lecture 
explanations of concepts and illustrated demonstrations (79% of all learning resources). The resources 
database also includes interactive simulations (1%) where students derive answers to problems by 
manipulating inputs to simulated functional systems, game-based activities (2%), assessments (0.34%), 
lesson plans (0.41%), audio (3%), and other (14%). Each video resource includes a description of the 
contents of the video with links to related resources. At the end of some videos, there is a link to an 
assessment tied to the content of the video. These assessments are separate from the ACT-based 
assessments (i.e., ACT Academy quizzes, ACT practice tests) featured in the Dashboard and Progress 
sections of ACT Academy. 


Progress 


The Progress section provides a detailed view of the quizzes available for each topic/reporting category 
with multiple quizzes available for each category. It also includes a tab for ACT practice tests. 


ACT Academy Quizzes 


The quizzes are short, consisting of 5-10 multiple choice questions. They are organized by the same 
topics used as reporting categories on the PreACT and ACT tests. See Appendix Table A1 for a list of the 
available quizzes. For each quiz in the Progress view, students see the date the quiz was taken, their 
score, their level of mastery (1 to 3 stars), a link to start the quiz (if quiz has not been taken), or links to 
retake the quiz or practice on questions answered incorrectly (if quiz has already been taken). 


Quiz items and learning resources are all tagged at the most granular level of the ACT Holistic 
Framework (Camara, O’Connor, Mattern, & Hanson, 2015) hierarchy-KSAOs (knowledge, skills, abilities, 
and other characteristics). ACT Academy utilizes the Recommendations and Diagnostics (RAD) API for 
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assessing at this level of each item in a quiz or practice test, and those estimates are propagated up the 
hierarchy to the reporting category level. ACT Academy utilizes this diagnostic calculation for determining 
level of mastery and relies on the RAD API to deliver personalized selections of learning resources. 


After completing a quiz, users are presented with a feedback screen displaying the percentage of 
questions answered correctly at the top. The questions are listed below along with an indicator of whether 
the response selected was correct. Additionally, the item responses are sent to the RAD API, and the 
correct/incorrect responses are incorporated into the student’s diagnostic model. These probabilistic 
values are used for determination of any updates to the star values displayed in Academy (cutoff values 
are applied). For incorrect responses, links are provided to resources that help users develop the targeted 
skills. Users can view the response options along with how they responded to a given question by clicking 
the “Show full Q&A” link. At this point, users can either view one or more of the recommended resources 
or they can close the results window to return to either the Dashboard or Progress screen to retake the 
same quiz or to select a different one. 


ACT Practice Tests 


In addition to tabs for each of the four subject areas showing the quizzes, the Progress section also 
includes a tab for ACT practice tests. Two practice tests are available for each subject area, with the 
number of questions matching a full-length ACT test (math = 60; science = 40; English = 75; reading = 
40). Practice tests use the same question formats and some of the same reading passages as quizzes, 
and the feedback provided to users at the end of practice tests is the same as for quizzes. The practice 
test tab shows a student's score on each practice test completed, using the 1-36 ACT score scale. These 
scores serve as a progress monitor and as a prediction for performance on an actual ACT test. 


Tips & Strategies 


The Tips & Strategies section provides a quick start guide for using ACT Academy, as well as several 
ACT test-taking strategies (e.g., pacing yourself, reading the directions for each question carefully, and 
answering every question). This section also includes tips specific to each subject area including 
descriptions of question formats and things to look for in each question (i.e. “be aware of questions with 
no underlined portions”). 


Assignments 


Students who are part of an Academy class will see an additional section called Assignments, which 
includes resources and assessments assigned by their teacher. Teachers can create assignments from 
scratch or based on a reporting category (e.g. ACT Algebra 1). For each assignment, students see the 
date it was assigned, the due date if one was set, and the completion status. 


Navigating ACT Academy 


In the quick start guide under Tips and Strategies, students are encouraged to develop their own path for 
navigating the assessments and learning resources offered by ACT Academy. General strategies for 
navigating ACT Academy could include: 


e Taking quizzes and/or practice tests and trying to identify misconceptions by delving into incorrect 
responses, without watching video lessons. 
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e Taking quizzes and/or practice tests to identify deficient skills and then watching video lessons 
that address those skills. 


e Scrolling through Resources and watching video lessons, with or without subsequently taking 
quizzes or practice tests. 


e Following the daily plan presented in Dashboard, where students are encouraged to practice at 
least 30 minutes per day, 2-3 days per week. 


e Completing assignments that are given by an instructor. 
e Some combination of the above approaches. 


In this study, we do not attempt to identify the strategies used nor try to understand which strategies are 
most effective. However, these are areas of study we recommend for future research. 


Current Study 


In this paper, we provide an early look at how students are using assessments within ACT Academy 
through the first 15 months of the platform’s existence and whether this usage results in higher ACT 
scores. Using a subset of the data generated by the platform, this initial investigation will form the basis 
for more extensive future research looking at patterns of use as they relate to ACT score gains and other 
learning outcomes. 


The ACT Academy platform records user performance when taking quizzes and practice tests and access 
and interaction with learning resources in the ACT Academy database. The current study is limited to 
assessment completion data, which includes a user identifier, start date, end date, and assessment name 
from which we can identify the subject area (English, math, reading, or science) and, for ACT quizzes, the 
reporting category (e.g., Knowledge of Language). Other assessment-related data, such as item 
responses and test scores, were not available to be used in this study. 


The study is limited to assessment use as the sole measure of engagement with the ACT Academy 
platform. Other aspects of engagement, such as use of video lessons, are not examined. Because ACT 
Academy quizzes and practice tests are important components of the learner experience that drive 
Academy’s recommendation and diagnostic engine, we believe that assessment use is a reasonable 
proxy for overall engagement with ACT Academy. 


The study addresses four research questions: 


e RQ1: How has usage of ACT Academy’s assessments trended over time? 
e RQ2: Does usage vary across different types of ACT Academy assessments? 


e RQ3: What are the demographic and academic characteristics of ACT Academy 
assessment users? 


e RQ4: Does use of ACT Academy assessments lead to higher ACT test scores? 
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Methods and Findings 


The study timeframe covers the launch of ACT Academy (March 2018) through the June 8, 2019 ACT 
test administration. There were over 211,000 ACT Academy user accounts, which include student, 
teacher, and parent accounts. Beginning with all ACT Academy accounts, Figure 1 shows how exclusion 
criteria are applied sequentially to arrive at the samples used to address the research questions. The 
exclusion criteria include: 


e Nouse of ACT Academy assessments (n=107,280). After this step, we have the sample used to 
address RQ1 and RQ2 (n=103,948). 


e No email address registered with an ACT Academy account (n=15,077). Email address is used 
in the next step to match to ACT test records. 


e No match to ACT test records (n=55,809). Students only have a match to an ACT test record if 
they took the ACT during the 2017-2018 academic year or later, provided an email address that 
was the same as the email address used for ACT Academy, and had an ACT test record that was 
not restricted for use.* For students who had an email address matching one or more ACT test 
records, we also attempted to obtain PreACT test records. The remaining sample is used to 
address RQ3 (n=33,062). Of note, the only source of data for demographic and academic 
characteristics of the sample is ACT test records (background data are not currently collected by 
ACT Academy). 


e No pre-post ACT tests with intervening use of ACT Academy assessments (n=19,049). A pre- 
post analysis design gives us greater confidence that we can measure the effect of ACT 
Academy assessment usage because students’ initial achievement level is controlled. While 
33,062 students have a matching ACT test record, only 14,013 students tested multiple times and 
used ACT Academy between the two tests.? 


e Nouse of other ACT test prep products (ACT® Online Prep, ACT® Kaplan® Online Prep Live) and 
no use of the practice test provided in the Preparing for the ACT booklet. By excluding students 
who are known to have used other test prep products, we can be more confident that any 
differences in ACT score gains are more likely due to ACT Academy usage and not other forms 
of test prep. After this step, we have the sample used to address RQ4 (n=9,287). 
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Figure 1. Exclusion Criteria Applied to Generate Samples for Addressing Research Questions 
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RQ1: How has usage of ACT Academy’s assessments 
trended over time? 


We addressed this research question (RQ1) by examining assessment completion data aggregated by 
week. As seen in Figure 2, usage typically spikes shortly before national test dates with the largest spikes 
occurring before the October, December, February, April, and June test dates. The steep drop after the 
December 8, 2018 test date suggests there is a lull in test preparation activity among students during the 
winter break and holiday season. The spikes in the number of assessments completed consistently seen 
in the final week or two before a test date suggest that many students are “cramming” for the ACT test. 


Figure 2. Number of ACT Academy Assessments Completed by Week 
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A comparison of spikes in activity (Figure 2) just before the April and June national test dates for 2018 
and 2019 suggest an approximate four-fold increase in Academy use over the past year.* For the April 
test dates, the number of assessments completed jumped 396% from 6,751 in April 2018 to 26,735 in 
April 2019. The same comparison for the June 2018 to June 2019 test dates saw the number of 
assessments completed increase by 415% from 6,937 to 28,758. 


In order to examine the overlapping months for the two years, we graphed the assessments completed by 
day (instead of by week as seen in Figure 2) for the April 1 to June 9 period for both 2018 and 2019 (see 
Figure 3 below). In addition to the magnitude of the spikes before national test dates, it is worth noting 
that the number of assessments completed on a daily basis for 2019 follows a similar pattern as is seen 

in 2018, though amplified considerably. Each peak represents the school week, and the valleys are the 
weekend. This may indicate that schools are the primary drivers of ACT Academy use, or that students 
are less motivated to use ACT Academy on the weekends. Additional research is needed to examine use 
by day and hour, which could inform how much Academy is being integrated into the school day. 


ACT Research & Policy | ACT Research Report | R1785 8 


Figure 3. Comparison of Number of ACT Academy Assessments Completed Between April 1 and June 9 
for 2018 and 2019 
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RQ2: Does usage vary across different types of ACT 
Academy assessments? 


We addressed this research question (RQ2) by presenting usage statistics for each ACT Academy 
assessment. Overall, there are 127 assessments, including eight practice tests (two for each subject 
area) and 119 quizzes (16 reporting categories, and 2-19 quizzes per reporting category). Appendix 
Table A1 lists the number of times each assessment was taken, the number of unique students who took 
each assessment, and the median assessment time. A total of 793,941 assessments were taken, with 
quizzes taken far more often than practice tests (Figure 4). 


Figure 4. Number of ACT Academy Assessments Taken, by Subject Area and Assessment Type 
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For each subject area, the number of quizzes taken dwarfed the number of practice tests taken. Students 
may have chosen quizzes over practice tests because they are much shorter. The median time to take a 
practice test ranged from 40.0 to 90.9 minutes (mean=57.2 minutes), while the median time to take a quiz 
ranged from 1.4 to 12.6 minutes (mean=4.6 minutes). 


The most quizzes were taken in math. There are a number of possible drivers for this. Perhaps more 
math quizzes are completed simply because math offers more quizzes (43) relative to English (37), 
reading (33), and science (7). Or, students may be more motivated to improve their math skills or may 
believe that math is more amenable to review and practice. It is also possible that since math is listed first 
in Academy (the tab opened by default in the Progress section), students just start with what is presented 
to them first. The number of practice tests taken was more consistent across subject areas, with the most 
practice tests taken in English. 


Usage varied extensively across the 119 quizzes, ranging from 817 for Conventions of Standard English, 
Quiz 12 to 58,484 for Math Number and Quantity, Quiz 8. From Table A1, we see that the quiz with the 
highest number for each reporting category was taken the most. For example, for Conventions of 
Standard English, Quiz 19 (the last quiz for this reporting category) was taken 16,786 times, whereas 
Quiz 1 was taken 5,867 times. Why is the last quiz for a reporting category being completed 
disproportionately more than others? Possibly, this is part of a review strategy: students are choosing or 
teachers are assigning the last quiz, believing that the quiz number (which is presented as a Roman 
numeral in Academy) indicates the quiz level, and students wish to test themselves at the highest level to 
determine if more work is needed. 


Another reason for variation in usage across assessments is that some assessments have been available 
longer on the ACT Academy platform. While ACT Academy was launched in March 2018, the second 
practice tests were first available in September 2018, and several quizzes were introduced in July 2018 
and September 2018. 


RQ3: What are the demographic and academic 
characteristics of ACT Academy assessment users? 


We addressed this research question (RQ3) using the sample of 33,062 students who took ACT 
Academy assessments and were matched to at least one ACT test record. Note that this is an 
underestimate of the actual number of students who used ACT Academy and took the ACT test because: 


e Not all students who used ACT Academy took an ACT Academy assessment, and so were not 
candidates for matching. 


e Not all students who took an ACT Academy assessment provided an email address. 


e Not all students who take the ACT test provide an email address (or may provide an email 
address different than what they use for ACT Academy), so cannot be matched. 


e Not all ACT test records are candidates for matching because of data use restrictions from 
contracts and state data privacy laws. 


In this section, we compare the Academy sample to the ACT-tested population, using the 2018 ACT- 
tested high school graduating class (ACT, 2018) as the population. We compare the Academy sample to 
the population on gender, race/ethnicity, parent education level, average ACT Composite score, average 
high school GPA, and school control (Table 1). 
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Table 1. Student Characteristics of 2018 ACT-Tested Graduating Class and Academy Assessment Users 
with a Matched ACT Test Record 


2018 ACT Academy Users 
Grad Class with ACT Match 


Characteristic 


Gender, % 
Female 51.8 64.4 
Male 46.7 35.5 
Missing 1.5 0.1 
Race/ethnicity, % 
African American 12.7 13.7 
Asian 48 11.4 
Hispanic 16.1 14.0 
Other 5.6 515 
White 52.1 51:1 
Missing 8.8 4.3 
Parent education level,> % 
No college (1-2) 16.1 14.4 
Some college (3-5) 18.5 18.5 
Bachelors (6) 21.9 27.5 
> Bachelors (7-8) 18.5 28.3 
Missing 25.0 11.4 
Average parent education level® 5.0 5.4 
Average ACT Composite score 20.8 23.4 
Average high school GPA 3:91 3.62 
School control, % 
Non-public 9.8 12.2 
Public 88.2 79.4 
Foreign 0.0 2.0 
Missing 2.0 6.4 


The 33,062 students who have used ACT Academy and have been matched to at least one ACT test 
record come from over 10,000 high schools across the United States and other countries. Relative to the 
ACT-tested population, the Academy sample has a larger concentration of female (64% vs. 52%), Asian 
(11% vs. 5%), and African American students (14% vs. 13%). The Academy sample has a smaller 
concentration of Hispanic (14% vs. 16%), White (51% vs. 52%), and students who did not provide their 
race/ethnicity (4% vs. 9%). 


Relative to the general population of ACT-tested students, Academy participants were more likely to 
report their parents’ education levels (89% vs. 75%). Among students who reported parent education 
level, it tended to be slightly higher for students in the Academy sample relative to the population. 


Academy students had higher academic achievement than the general population of ACT-tested 
students. Mean ACT Composite score (using most recent score) was 23.4 for the Academy sample, 
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compared to 20.8 for the population. Mean high school GPA, based on self-reported course grades, was 
3.62 for the Academy sample and 3.31 for the population. 


While most Academy students (79%) attended public high schools in the United States, relative to the 
population, they were more likely to attend non-public schools (12% vs. 10%) or attend a high school with 
unknown affiliation (6% vs. 2%). A small share of Academy students (2%) attended high school outside 
of the United States. Appendix Table A2 lists the number of identified ACT Academy users with ACT test 
records, by state. The states with the most ACT-tested Academy users include Texas (n=2,371), Florida 
(n=2,270), California (nN=1,749), Tennessee (n=1,569), and Ohio (n=1,507). As expected, the number of 
ACT-tested Academy users in a state is highly correlated with the number of ACT-tested high school 
graduates in a state (r= .89). 


RQ4: Does use of ACT Academy assessments lead to higher ACT test 
scores? 


We addressed this research question (RQ4) by comparing ACT score gains for Academy users to those 
of similar students who did not participate in ACT Academy. This analysis is restricted to the 9,287 
students who participated in ACT Academy assessments between PreACT/ACT tests, in addition to a 
large comparison group of students from the same high schools who did not use ACT Academy. To be in 
the comparison group, students must have: 


e Taken the ACT atleast twice between April 2018 and June 2019, with at least one month 
between tests. Note that the PreACT test could be used in place of the ACT for the pretest. 


e Nouse of ACT Online Prep (AOP), ACT Online Prep Live (OPL), or ACT Academy using an 
account with the same email address used to register for the ACT. Note that these exclusion 
criteria cannot be executed with 100% accuracy because some students may have used ACT 
test prep resources with a different email address than that used for the ACT test. 


e Must not have responded “Yes” to two questions regarding test preparation prior to taking the 
ACT: “Did you prepare for the ACT test using any test preparation materials (for example, The 
Official ACT Prep Guide, other study guides, online materials, practice tests, tutors, or test prep 
courses)?” and “Did you prepare for today's test using ACT's free, full-length practice test found in 
the Preparing for the ACT booklet?” Note that students who are missing responses to these 
questions, which are only asked during national ACT test administrations, are not excluded. 


e Must have attended the same high school as at least one of the ACT Academy participants. 


Overall, 326,599 students met these criteria and are included in the comparison group. 


Measuring usage of ACT Academy assessments 


Level of usage of ACT Academy assessments can be measured multiple ways, including total number of 
assessments taken, total amount of time spent on assessments, total number of quizzes taken, total 
number of practice tests taken, and total number of days with at least one assessment. We chose to 
focus on the amount of time spent on assessments because we thought it was a reasonable proxy 
measure for overall commitment to, and engagement with, ACT Academy. Time spent on assessments 
self-weights assessments with different time commitments. For example, students who take a practice 
test generally spend more time than students who take a quiz and thus should have a greater measure of 
assessment usage. 
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For each assessment event, our data set included start and end times, which we used to calculate 
assessment duration. In many cases, students had assessment events that were unrealistically long (e.g., 
a practice test that took two weeks to complete). In these cases, the student likely paused the 
assessment and completed it on a later date, and so the assessment duration was not an accurate 
measure of their time on task. Therefore, we could not simply use the sum of all assessment durations as 
the measure of total assessment time. Instead, we first identified a reasonable upper bound for the 
duration of each type of assessment. Then, if a student's assessment time exceeded the upper bound, 
the assessment time was considered an outlier, and we only credited the student with the upper bound as 
their assessment time. 


To calculate an upper bound of assessment time for each assessment, we followed these steps: 


1. Calculate the median (P50) assessment time. 


2. Among assessments completed within the median assessment time, calculate the truncated 
standard deviation (TSD) of assessment time. 


3. Using the truncated standard deviation, estimate what the standard deviation would be for the 
entire distribution, under the assumption that the left side of the distribution (below the median) is 
symmetric to the right side of the distribution, and that the total distribution is normal. The formula 
for this modified standard deviation is MSD = TSD*sqrt(t/(11-2)) (Greene, 2003). 


4. Outliers are identified as assessment times that are greater than P50 + 2*MSD. 


Conceptually, this procedure for identifying outliers is the same as flagging values that are more than two 
standard deviations above the simple mean. But because the simple mean and standard deviation are 
very sensitive to severe outliers in assessment time, we used the steps above to use statistics (P50 and 
MSD) that are not sensitive to severe outliers in assessment time. 


After calculating assessment duration for each student and each subject area, we inspected the 
frequency distributions. We found that many of the Academy participants had only used assessments in 
one subject area. Among the 9,287 participants, 48% were assessed on English, 67% on math, 29% on 
reading, and 39% on science. We sought a categorization of assessment hours that would result in a 
reasonably large number of students in each category. 


For each subject area, we categorized assessment duration into five levels: 0 (no assessment usage), 
<0.5 (some assessment use, but less than half an hour of assessments), 1 (0.5 to 1.5 hours of 
assessments), 2 (1.5 to 2.5 hours of assessments), and 3+ (more than 2.5 hours of assessments). Figure 
5 provides the number of Academy participants in each assessment use level. The groups shown in 
Figure 5 can be thought of as treatment groups with different dosage levels. As dosage level increases, 
the number of students decreases. The number of students in the highest dosage groups are relatively 
small, especially for science (n=59) and reading (n=68). 


Number of Students 


Number of Students 
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Figure 5. Number of Students in RQ4 Sample, by Subject Area and ACT Academy Assessment Hours 
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Taking a step back to the sample used to examine RQ1 and RQ2 (n=103,948), we can examine the size 
of the participant groups for all students who have used ACT Academy assessments (Figure 6). We see 
that the levels of use among the population of ACT Academy users are similar to those of the restricted 
RQ4 sample. For the RQ4 sample, the percentage of students with at least one hour of assessment use 
was 15% for English, 26% for math, 9% for reading, and 13% for science. For the population of Academy 
users, the percentage of students with at least one hour of assessment use was 14% for English, 23% for 
math, 9% for reading, and 13% for science. 


Figure 6. Number of ACT Academy Participants, by Subject Area and ACT Academy Assessment Hours 
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Next, we calculated the average ACT score gain (posttest score — pretest score) for each group of 
Academy participants, as well as for the comparison group. Figure 7 shows the average gain scores, and 
the black bars represent 95% confidence intervals for the average gain. Note that the confidence intervals 
are rather wide for the higher-dosage groups, especially for those in the 3+ hours group, because the 
sample sizes are relatively small. 


From Figure 7, we see that average gain scores are largest in English (1.52 for the non-Academy 
comparison group) and smallest in reading (0.67 for the non-Academy comparison group). ACT scores 
generally increase with more ACT Academy assessment time. Exceptions are observed for reading, 
where the mean gain for the one-hour group is slightly lower than the mean for the <0.5-hour group, and 
for science, where the mean gain for the one-hour group is greater than the mean gains for the two-hour 
and 3+ hour groups. For each subject area, the average gain for the two-hour group is around twice that 
of the comparison group. 


Figure 7. Average ACT Gain Scores, by Subject Area and ACT Academy Assessment Hours 
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The results shown in Figure 7 do not account for differences between the ACT Academy participants and 
the comparison group that could influence the choice to use ACT Academy, as well as ACT score gains 
(e.g., demographics, prior academic achievement, time between pre- and post-test). Table 3 compares 
the ACT Academy participant groups for English to the comparison group on several covariates. 
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Table 3. Student Characteristics of Comparison Group and ACT Academy English Participant Dosage 
Groups 


ACT Academy Assessment Hour Group: 


Comparison 


Variable Group English 
Gender, % 
Female 52.4 66.0 63.6 63.8 66.6 66.2 
Male 47.6 34.0 36.4 36.2 33.4 33.8 
Race/ethnicity, % 
African American 12.1 12.4 TAT 12.1 6.6 12.8 
Asian 4.5 8.5 7.5 11.6 14.5 15.0 
Hispanic 10.7 11.2 10.6 10.9 12.7 11.3 
Other 6.1 6.3 6.0 5.4 Sf 8.3 
White 63.6 58.9 61.5 57.5 56.0 50.4 
Missing 2.9 2.7 2 2.5 4.5 2.3 
Parent education level, % 
No college (1-2) 12.9 13.5 13.1 11.8 13.9 12.0 
Some college (3-5) 17.0 18.3 20.1 19.2 18.4 16.5 
Bachelors (6) 25.8 29.8 30.2 29.3 29.8 27.8 
> Bachelors (7-8) 21.0 30.2 26.9 31.9 31.9 33.8 
Missing 23.2 8.2 97 7.8 6.0 9.8 
eWelage DicleskGempesite 20.6 227 «=PiT 22% 339 22.1 
score 
High school GPA, % 
<3.0 14.2 8.8 9.4 8.3 6.6 6.0 
3.0-3.5 15.8 15.1 15.9 15.5 9.6 18.0 
3.5-4.0 38.1 63.3 58.5 65.1 70.5 61.7 
Missing 31.9 12.9 16.2 11.1 13.3 14.3 
Average HSGPA 3.45 3.65 3.62 3.66 3.73 3.68 
eyeiads Mee patie 9.5 77 82 84 8.9 10.0 
pretest/posttest 
Pretest type, % 
ACT national 40.0 60.1 55.9 58.8 60.2 60.9 
ACT state/district 14.5 16.7 16.6 14.7 a ale 9.0 
PreACT 45.5 23:2 27.6 26.5 28.0 30.1 
Posttest type, % 
ACT national 46.7 77.4 69.3 76.9 76.5 73.7 
ACT state/district 53.3 22.6 30.7 2331 23.5 26.3 
Test accommodations, % 
No 92.6 97.4 96.2 97.3 98.2 95.5 
Yes 7.4 2.6 3.8 2.0 1.8 4.5 


From Table 3, we see that the ACT Academy participant groups are different than the comparison group 
on several variables. Academy participants are more likely to be female, Asian, have higher parent 
education level, have higher HSGPA, and to have tested on an ACT national test date for their pretest 
and posttest. Academy participants are less likely to be White, to have taken the PreACT as their pretest, 
and to have had a testing accommodation for their ACT posttest. There are also some differences across 
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the Academy participant dosage groups, but the differences are generally small. Table 3 only shows the 
comparison for English; the comparisons are similar for the other subject areas. 


The comparison group and ACT Academy participant groups are different in ways that could confound 
comparisons of ACT score gains. We used a propensity score weighting technique (Austin, 2011) to 
achieve covariate balance between each participant group and the comparison group. Specifically, we 
took these steps: 


e For each Academy participant group (i.e., each treatment dosage group) and each subject area, 
model the probability of being in the treatment group relative to the comparison group. Logistic 
regression was used to model treatment group status (students who used Academy) using the 
following predictor variables: gender, race/ethnicity (African American, White, Hispanic, Asian, 
Other, missing), parent education level (<high school, some college, bachelor’s, more than 
bachelor’s, missing), Composite pretest score, high school GPA (<3.0, 3.0-3.5, 3.5-4.0, missing), 
number of months between pretest and posttest, pretest type (PreACT, ACT national, ACT 
state/district), posttest tyoe (ACT national or ACT state/district), and whether the student tested 
with special accommodations on the ACT posttest. The logistic regression model estimates the 
probability of being in the treatment group, which is used as the propensity score (ps). 


e For students in the comparison group, assign a weight (wt) calculated as wt = ps/(1-ps). For 
students in the Academy participant group, set wt = 1. By weighting the comparison group in this 
manner, we create a synthetic sample in which the distribution of covariates is independent of 
group (comparison vs. Academy participant group; Austin, 2011). 


e The group difference in the weighted mean of the outcome (in our case, ACT gain score) 
represents the average treatment effect for the treated (ATT; Imbens, 2004). In our case, the ATT 
is the average effect of participating in ACT Academy assessments for students who participated 
(Austin, 2011). 


These steps were performed separately for each participant dosage group (0, <0.5, 1, 2, 3+) and each 
subject area. After weighting, the comparison group is similar to each participant dosage group on the 
variables included in the propensity score model. For English, Table 4 shows the summary statistics for 
the weighted comparison group, as compared to each participant dosage group. Because members of 
the treatment group receive a weight of 1, the statistics in Table 4 for the treatment group are identical to 
the unweighted statistics (Table 3). 
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Table 4. Student Characteristics of Comparison Group and ACT Academy English Participant Dosage 
Groups, after Propensity Score Weighting 


ACT Academy assessment hour group: English 


Variable 


Gender, % 
Female 
Male 


Race/ethnicity, % 
African American 
Asian 
Hispanic 
Other 
White 
Missing 


Parent education level, % 
No college (1-2) 
Some college (3-5) 
Bachelors (6) 
> Bachelors (7-8) 
Missing 
Average pretest 
Composite score 


High school GPA, % 
<3.0 
3.0-3.5 
3.5-4.0 
Missing 
Average HSGPA 
Average months between 
pretest/posttest 
Pretest type, % 
ACT national 
ACT state/district 
PreACT 


Posttest type, % 
ACT national 
ACT state/district 
Test accommodations, % 
No 
Yes 


Note: T = treatment group, C = comparison group. 
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While Figure 7 shows the unweighted mean ACT gain score for each group, Figure 8 (below) shows the 
difference in weighted mean ACT gain scores for each Academy participant group relative to the 
comparison group. The black bars represent 95% confidence intervals for the effect estimates. If the 
black bars encompass 0, the effect is not statistically significant. 


Figure 8. Estimated Effects of ACT Academy Assessment Use on ACT Scores, by Subject Area 


1.90 


ACT Score Effect Estimates 
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For example, consider the effects of participating in ACT Academy reading assessments for one hour 
(blue bar). From Figure 7, the unadjusted average ACT reading score gain is 1.38 for students who 
completed one hour of Academy Reading assessments and is 0.67 for the comparison group. Therefore, 
a raw estimate of the effect of participating in one hour of reading assessments is 0.71 (1.38-0.67) ACT 
score points. Figure 8 shows that the corresponding adjusted (weighted) mean gain score estimate is 
0.66, with a 95% confidence interval of [0.34, 0.99]. In most cases, the raw effect estimates (derived from 
Figure 7) are similar to the adjusted effect estimates (Figure 8). Table 5 presents the raw and adjusted 
effects, along with standard errors and tests of statistical significance. 
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Table 5. Raw and Adjusted Effects of ACT Academy Participation 


Raw Effect Adjusted Effect 
Subject Estimate Estimate 


Comparison 


0-hour 4,800 <.001 
English <O-Sshour 3,049 <.001 
g 4-hour 973 <.001 


<.001 
<.001 


2-hour 332 
3+ hour 133 
Comparison 


0-hour 3,028 .008 


Math <0.5-hour 3,832 <.001 
1-hour 1,587 <.001 
2-hour 400 <.001 
3+ hour 440 <.001 


Comparison 326,599 


0-hour 6,610 

. <0.5-hour 1,819 
Reading 1-hour 657 
2-hour 133 

3+ hour 68 

Comparison 326,599 

O-hour 5,625 

eae <0.5-hour 2,417 
1-hour 1,002 

2-hour 184 


3+ hour 59 


Note: EST = estimate, SE = standard error 


In all cases, the estimated effect of participation in ACT Academy assessments was positive, and in all 
cases but two (Science, O-hour group and 3+ hour group) was statistically significant (p < .05). The effect 
estimates suggest that increased use of ACT Academy assessments results in larger ACT score gains. 
Students who spend an hour on ACT Academy assessments in one subject area can expect ACT subject 
score gains that are 0.6 to 1.3 points higher than if they had spent no time on ACT Academy 
assessments. 


Students in the lowest dosage group (0-hour) participated in at least one assessment but none for the 
subject area in question. If the propensity score weighting method was effective (e.g., removed group 
differences in factors that may relate to ACT score gain), one might expect ACT score gains to be similar 
for the 0-hour and comparison groups because neither group had any known engagement with ACT 
Academy for the subject area in question. From Figure 7, we see that the 0-hour group had slightly higher 
ACT score gains in English, math, and reading. Possible explanations for this finding include: 
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e Students in the 0-hour group benefitted from non-assessment ACT Academy activities that were 
not tracked in this study (e.g., video resources, test-taking strategies). 


e Students in the 0-hour group benefitted from participation in ACT Academy assessments from 
other subject areas. In other words, there could be spill-over effects where work in one subject 
area has a small positive effect on performance in other subject areas. 


e The propensity score weighting procedure did not remove all baseline differences between 
participants and the comparison sample, and the participant groups show larger ACT score gains 
due, in part, to unmeasured factors (e.g., motivation to achieve a higher ACT score). 


Effects on ACT Composite score 


Up to this point of the analysis, we have focused on effects of ACT Academy assessment use on ACT 
subject test scores. However, students are most interested in increasing their ACT Composite score 
because it measures overall performance and is commonly used for college admissions and 
scholarships. Among the 9,287 students who participated in at least one assessment, the average 
Composite score gain was 1.33, compared to 0.98 for students in the comparison group. Thus, if we did 
not consider different levels of assessment dosage and differences in baseline covariates, we would 
conclude that use of Academy assessments leads to a 0.35-point increase in ACT Composite score. 


We calculated total assessment time by summing the assessment durations across the four subject 
areas. After inspecting the frequency distribution of total assessment time, we grouped students into six 
dosage groups based on total assessment time: <0.5, 1 (0.5-1.5 hours), 2 (1.5-2.5 hours), 3-5 (2.5-5.5 
hours), 6-10 (5.5-10.5 hours), and 10+ (>10.5 hours). We then used the same propensity score weighting 
methods described earlier to estimate the effects of different levels of assessment participation (Table 6). 


Table 6. Effects of ACT Academy Participation on ACT Composite Score by Dosage Level 


Raw Effect Adjusted Effect 
Total Assessment : : 
Estimate Estimate 
Hours 


Comparison 


<0.5 4,861 
1 2,470 

2 843 
3-5 779 
6-10 265 
>10 69 


Note: EST = estimate, SE = standard error 


Consistent with the subject-specific results, we find that ACT Composite scores increase with additional 
time spent on ACT Academy assessments. From Table 6, a student who spends 1.5-2.5 hours on 
assessments in each subject area (e.g., 6-10 hours total) could be expected to earn an ACT Composite 
score of about 1.29 points higher than if they had not spent any time on Academy assessments. This 
result is similar to what we would estimate from the subject-specific results: From Table 5, we would 
expect an effect of 1.13” points on ACT Composite score following two hours (1.5-2.5 hours) of 
assessment in each subject. The estimated effect of spending 3-5 hours on assessments is 0.70 ACT 
Composite score points (Table 6), which is smaller than the estimate derived by averaging the subject- 
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specific effects related to one hour of use (0.90). One reason for this discrepancy is that students tend to 
accumulate more assessment hours in math, and the math effects are somewhat smaller than the effects 
for the other subject areas. 


Discussion 


Since its launch on March 22, 2018, ACT Academy has seen strong growth in the number of registered 
users and up to a 415% increase in user activity as measured by assessments completed between April 
and June for 2018 and 2019. As evidenced by the spikes in assessment usage before national ACT test 
administrations, students commonly use ACT Academy in close time proximity to the test (“cramming”). 
Usage also appears to be more concentrated on weekdays, suggesting that ACT Academy is 
implemented as a component of school-based test prep classes; in subsequent analyses, we plan to 
delve more deeply into this question. Consistent with our findings, a recent study (Moore, Sanchez, San 
Pedro, & Payne, 2019) found that schools in states with State and District testing are using advisory 
periods for test preparation at least one day per week for several months prior to the spring ACT test. 


Taking quizzes is the dominant assessment activity in ACT Academy coupled with math as the dominant 
content area of focus; students complete almost as many assessments in math as the other three 
subjects combined. Relative to the ACT test-taking population, ACT Academy users are more likely to be 
high-achievers, female, Asian, attend non-public schools, and have parents with a higher education level. 
It was not surprising that states with more ACT-tested students tend to have more ACT Academy users, 
including some states with ACT State and District testing programs. 


While relatively few students completed two or more hours of assessments in ACT Academy in each 
subject area, effects of assessment use on ACT gain scores are practically and statistically significant. 
After implementing propensity score weighting to achieve balance between ACT Academy participant 
dosage groups (0, <0.5, 1, 2, 3+ hours of assessments) and a comparison group, we see subject gain 
scores ranging from 0.61 to 1.32 points for students spending one hour completing assessments, 0.80 to 
1.66 points for students spending two hours, and 0.91 to 1.90 points for students spending three or more 
hours. The prospect of increasing ACT scores by 1-2 points with what appears to be a reasonably short 
time commitment suggests a clear benefit to students. 


Among all ACT Academy participants, the unadjusted average ACT Composite score gain was 1.33, 
compared to 0.98 for the comparison sample, suggesting a modest benefit of just 0.35 ACT Composite 
score points. We found that the effectiveness of ACT Academy depends heavily on how much time 
students spend on assessments. For example, students who spent approximately eight hours on 
Academy assessments saw, on average, an incremental gain in their ACT Composite score of 1.3 points. 
These results are aligned with previous research on the impact of preparation activities on score gains 
(Bangert-Drowns, Kulik, & Kulik, 1983; Montgomery & Lily, 2012; Powers, 1993; Appelrouth, et al., 2017; 
Moore, et al., 2018). Schiel and Valiga (2014a, 2014b) found a relationship between time invested in test 
preparation and ACT score gains. Students spending over 20 hours had an unadjusted average 
Composite score gain of 1.7, compared to 0.8 for students with no test prep. In a study of group and 
individual tutoring for the SAT, Appelrouth, Zabrucky, and Moore (2017) found that approximately 13 
hours of tutoring increases SAT total score by 30 points (.25 SD), which is similar to a 1.4 point increase 
in ACT Composite score. A study of ACT Online Prep (Sanchez, 2019) found an incremental relationship 
between time spent using the tool and ACT Composite score gain with a difference of 0.44 points in ACT 
Composite score for those students using ACT Online Prep for 21+ hours (1.59 points) versus those who 
didn’t use ACT Online Prep at all (1.16 points). 
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In comparison with some of these past studies, we found the same magnitude of an effect on the 
Composite score with only eight hours of time completing assessments. However, it is important to 
include the caveat that we were not able to include time spent reviewing the learning resources and other 
activities available to students in ACT Academy due to the limitations of the data set available. Time 
spent on assessments is only a proxy measure of overall engagement with ACT Academy, and additional 
research is needed to understand how time spent on assessments relates to total time spent in ACT 
Academy. It is possible the effects corresponding to the total time spent in ACT Academy may be 
comparable to the effect-time correspondence reported in previous studies. 


There are two aspects to efficacy or effectiveness that are important to students and educators: One is 
improving ACT scores on subsequent tests, and the other is efficiency - the time required to achieve 
those score gains. Efficiency, at its core, is a return-on-investment argument. If ACT Academy can 
achieve the same score gains with less time and expense than other products or approaches, this is a 
significant advantage, particularly if we can demonstrate empirically that for a given increment of time 
spent actively engaged with the platform, students can expect a corresponding increase in their test 
scores. 


Study Limitations 


As mentioned previously, this study was limited to assessment 
completion data for both quizzes and practice tests and did not include 
any data that would provide a measure of time spent on learning 
resources (e.g., video lessons). Without these data, we cannot 
measure total time spent in Academy and can’t make claims of 
superior efficiency of ACT Academy; however, the early indicators 
from this study look promising in this respect. Additional data will also 
help us measure engagement with ACT Academy in more nuanced 
ame ways, identify navigation strategies, and test if some strategies are 
more effective than others. 


We attempted to eliminate the confounding effect of other forms of test preparation through the execution 
of exclusion criteria. We excluded students who indicated using other forms of test prep, as well as 
students known to have used ACT Online Prep and/or Online Prep Live. However, limitations of the data 
used to execute these exclusion criteria limit our ability to completely exclude students who used other 
forms of test prep. Further, it is possible that students who used other resources or had participated in 
school-based test-prep courses responded “No” to the questions regarding other forms of test prep. Most 
interviewees in a qualitative implementation study of ACT Online Prep (Moore, et al., 2019) indicated that 
in states where the ACT is required and used as an accountability measure, students may receive up to 
one hour per week of test preparation, both in terms of content instruction and test-taking strategies, for 
up to six months prior to taking the ACT during spring of their junior year. As a result, students from states 
with State and District testing may have answered “No” even though they received months of test prep 
instruction at school. 


There are also limitations inherent in the observational, quasi-experimental design of this study. Without 
the ability to assign participants to specific treatment conditions, we cannot rule out the possibility that 
unobserved factors contribute to the differences we observed. 
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Future Research 


Since this initial study only examines one aspect of Academy use (assessments), the scope of our inquiry 
was limited. Once a more comprehensive data set is available for research, we can begin studying the 
impact of ACT Academy on test preparation and learning by examining patterns of learner interaction with 
the platform as they relate to performance on assessments within Academy and score gains on the ACT. 
An important element to study is the use of learning resources and what viewing patterns might tell us 
about how students approach test prep. For instance, do students first view learning resources after 
incorrectly answering quiz questions or do they view suggested learning resources first before taking a 
quiz, as would be the case if they follow the daily plan? Do students watch videos to completion, or does 
it depend on the video’s length? Do students scrub through a video, jumping around looking for specific 
content, or do they watch more sequentially from start to finish? Is there are relationship between 
assessment item response and assessment completion patterns and learning resource usage? Do the 
patterns of high-performing students look similar and are they distinct from lower-performing students? 


The study used a pretest-posttest design and so only examined effects of ACT Academy usage for 
students who had already taken the ACT (or PreACT). It is possible that the effects of ACT Academy are 
different for first-time examinees who have not had prior exposure to the test and so may have more to 
gain in terms of familiarity with test format, test-taking strategies, and test content. Future studies should 
examine effects for first-time examinees. Similarly, future studies should examine whether the effects of 
using ACT Academy are similar for different subgroups of students, defined by characteristics such as 
baseline academic achievement level, race/ethnicity, gender, and socioeconomic status. 


Combining research on learner response and usage patterns with more qualitative implementation 
studies of the school and home settings where students use Academy can provide valuable insight into 
how contextual affordances and constraints impact usage and outcomes. These insights could inform 
design decisions to improve the capabilities of the platform and what types of marketing outreach might 
be most beneficial. As Figure 3 suggests, the bulk of user activity is concentrated on weekdays. This 
would suggest that Academy is being integrated into school-based test preparation activities. It would be 
useful to better understand how schools are using Academy: as an independent, self-study tool or are 
teachers selecting specific assessments and learning resources and using the Assignment function to 
create a custom test prep curriculum for their students? What percentage of students who are introduced 
to Academy at school continue to use it at home? 


The current study found a significant effect of ACT Academy usage on ACT test scores. The effect was 
largest for English, suggesting that the ACT English test is more amenable to practice and preparation or 
that ACT Academy’s learning resources are relatively more effective for English. Additional research is 
needed to understand why the effect of using ACT Academy is stronger for English. Future studies could 
also examine if the benefits of using ACT Academy transfer to external outcomes, such as performance 
in high school and college courses, and skills, such as critical thinking, writing, and scientific reasoning. If 
it does lead to positive transfer of knowledge and skills, it would indicate broader effects on learning. 


In conclusion, understanding ACT Academy as both a learning platform and test prep resource could help 
inform future development of the platform in terms of content, features, and functionality (i.e. more robust 
and differentiated corrective feedback, more scaffolding in activities, collaborative or peer instruction 
modes, game-based elements, etc.). Findings from this initial investigation of the effectiveness of ACT 
Academy show promising relationships between time spent on assessments and gain scores on the ACT; 
however, this is merely the beginning of an exploration of the potential of ACT Academy as a learning 
platform. As more data become available for research, increasingly compelling questions may be 
explored. 
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Notes 


1. The reporting categories also correspond to the domains used to group the Common Core State 
Standards (www.corestandards.org) for reading, language, and math. 

2. Some ACT state and district testing records cannot be used for this study due to contractual 
obligations and/or state data privacy laws. 

3. Either PreACT or ACT test scores were used as pretest scores, and ACT test scores were used as 

posttest scores. 

ACT Academy went live on March 22, 2018. 

Based on highest of mother’s and father’s education levels. 

Average based on highest of mother’s and father’s education levels, using the 1-8 ordinal scale. 

1.13 is derived as the average of the subject-specific effects (1.66 for English, 0.80 for math, 0.93 for 

reading, 1.13 for science) for the 2-hour groups. 
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Appendix 


Table A1. ACT Quizzes and Practice Tests Available in ACT Academy 


Assessment name Form N Exams N Students Median Time 


: : 1 7,983 7,201 57.4 
English practice test 2 2,080 1,819 58.6 
, 1 5,941 5,305 79.7 

Math practice test 2 2,383 1.994 90.9 
: ; 1 5,951 5,341 44.1 
Reading practice test 2 1.637 1,415 40.0 
Science practice test : re Geer — 
2 2,327 2,055 44.4 

| 5,855 4,551 Sul 

II 3,890 2172 4.3 

Il 2,796 2,133 4.3 

IV 2,550 1,814 5.7 

Vv 4,218 3,354 5.4 

Vi 1,284 1,032 4.2 

Vil 1,048 913 3:7 

Vill 955 822 3.4 

IX 881 793 3.5 

English: Conventions of standard English X 848 743 4.1 
Xl 835 712 4.0 

xil 814 686 4.1 

XIll 837 704 4.1 

XIV 867 722 3.8 

XV 873 748 3.4 

XVI 954 792 2.5 

XVII 1,098 903 2.8 

XVIII 1,467 1,272 4.5 

XIX 16,756 13,127 4.8 

| 9,368 7,186 £3 

II 3,396 2,784 2.3 

Il 2,879 2,428 2:5 

: IV 2,530 2,074 1.4 
English: Knowledge of language Vv 2 349 2.015 17 
Vi 2.272 1,960 2.0 

Vil 2,700 2,324 2.3 

VIII 29,760 18,612 2.2 

| 15,745 12,455 10.0 

I 10,838 8,794 12.4 

Ill 5,565 4,291 4.3 

IV 4,787 3,732 3.2 

English: Production of writing V 4,913 3,444 2.4 
VI 4,869 3,267 2.1 

Vil 4,179 3,274 2.2 

VIII 5,433 3,984 2.9 


IX 37,802 24,998 2.8 
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Assessment name Form N Exams N Students Median Time 
| 11,963 9,665 9.3 
I 6,101 4,074 3.6 
Il 4,458 2,960 3.3 
IV 3,646 2,511 2.4 
; V 3,373 2,312 1.7 
Many cigena Vl 3,046 2,245 3.1 
Vil 2,843 2,096 3.6 
Vill 2,724 2,024 3.5 
IX 3,262 2,442 3.9 
x 34,136 23,080 6.0 
| 7,415 6,150 6.2 
II 3,831 2,430 4.3 
Math: Functions Il 2,754 1,862 23 
IV 2,524 1,765 2.5 
V 14,441 10,331 3:3 
| 6,013 5,080 11.2 
H} 4,720 3,595 11.0 
HT 2,129 1,562 4.8 
IV 1,774 1,325 3.8 
Math: Geometry V 1,688 1,262 o2f 
Vi 1,614 1,191 3.9 
Vil 1,578 13112 3.5 
Vill 1,839 1,381 3.8 
IX 18,125 135275 6.4 
| 3,920 3,307 74 
I 3,816 2,799 6.2 
Math: Integrating essential skills Il 1,846 1,377 4.8 
IV 1,717 13255 4.3 
V 6,684 4,901 Tal 
| 6,009 4,364 5.1 
Math: Modeling H} 2,319 1,726 4.3 
Ill 8,310 5,938 3.2 
| 26,922 20,364 12.6 
H} 13,141 9,289 3.1 
Il 9,870 7,503 4.8 
; : IV 7,882 5,958 6.6 
Math: Numbers and quantity Vv 3,345 2.499 55 
Vi 6,326 4,617 4.1 
VII 7,218 5,313 5.8 
VIII 58,484 38,000 5.1 
| 7,308 5,469 8.4 
Math: Statistics and probability Il 3,433 2,536 4.4 


Il 15,087 11,331 4.9 
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Assessment name Form N Exams N Students Median Time 
| 3,403 2,440 4.6 
Il 4,454 3,773 8.3 
Il 1,230 913 3.6 
IV 985 680 5.0 
; V 964 660 2.1 
Reading: Craft and structure vi 925 747 3.3 
VII 868 646 2.4 
VIII 892 623 2:7 
IX 1,348 1,035 3.3 
x 14,556 9,247 3.1 
| 3,476 2,652 7.1 
Il 1,412 942 27 
Il 1,195 769 2:3 
: : : IV 939 681 2.4 
Reading: Integration of knowledge and ideas V 909 690 28 
Vi 912 630 2.5 
Vil 1,031 746 3.5 
IX 10,832 6,976 3.1 
| 8,979 6,837 10.6 
II 4,651 3.539 Do 
Il 4,231 2,968 6.3 
IV 4,754 3,624 TH 
V 2,188 1,669 3.2 
Vi 1,872 1,408 4.9 
Vil 1,636 1,221 5A 
Reading: Key ideas and details Vill 1,632 1,131 3.2 
IX 1,527 1,036 3.9 
Xx 1,748 1,083 2.7 
Xl 1,457 1,072 3:7 
Xll 1,674 1,145 3.1 
XIll 2,114 1,369 3.0 
XIV 2,488 1,857 3:3 
XV 21,250 14,630 4.6 
Science: Evaluation of models, inferences, | 6,353 4,643 6.9 
and experimental results II 11,427 8,395 8.5 
| 13,987 11,480 f2 
Science: Interpretation of data I 10,330 7,969 5.5 
HT 38,130 27,045 6.7 
| 8,489 6,248 5.6 


Science: Scientific Investigation r 19.283 13,724 73 
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Table A2. ACT-Tested ACT Academy Users, by State 


ACT-tested ACT-tested graduates Users per 100 

Academy Users Co) WAU Ks} graduates 
Alabama : 
Alaska 48 2,582 1.86 
Arizona 685 45,468 {| 
Arkansas 679 34,468 1.97 
California 1,749 118,521 1.48 
Colorado 238 17,081 1.39 
Connecticut 167 10,534 1.59 
Delaware 29 1,654 1.75 
District of Columbia 23 1,748 1:32 
Florida 2,270 119,543 1.90 
Georgia AAS) 56,481 PAKS) 
Hawaii 129 12,460 1.04 
Idaho 140 302 1.90 
Illinois 858 62,626 A -S0/ 
Indiana 345 23,580 1.46 
lowa 544 24,028 2.26 
Kansas 520 25,103 2.07 
Kentucky 951 51,814 1.84 
Louisiana 1,258 55.711 2.26 
Maine 16 1,043 153 
Maryland 342 19,408 1.76 
Massachusetts 329 18,219 1.81 
Michigan 417 22,738 1.83 
Minnesota 696 GileZ53) 1.14 
Mississippi 916 37,654 2.43 
Missouri 951 68,424 1.39 
Montana 188 9,731 1.93 
Nebraska 329 24,516 1.34 
Nevada 540 35,865 1.51 
New Hampshire 38 2,440 1.56 
New Jersey 466 32,590 1.43 
New Mexico PaaS) 13,749 1.62 
New York 870 56,340 1.54 
North Carolina 632 109,256 0.58 
North Dakota 105 7,282 1.44 
Ohio 1,507 127,392 1.18 
Oklahoma 1,372 42,388 3.24 
Oregon 184 15,432 1.19 
Pennsylvania 563 27,694 2.03 
Rhode Island 26 1,554 1.67 
South Carolina 411 53,743 0.76 
South Dakota 186 6,730 2.76 
Tennessee 1,569 79,170 1.98 
Texas P23 | 141,253 1.68 
Utah Sil 43,791 0.71 
Vermont 41 1,633 2.51 
Virginia 575 21,645 2.66 
Washington 330 17,010 1.94 
West Virginia 170 11,601 1.47 
Wisconsin 716 68,000 1.05 


Wyoming 188 6,342 2.96 
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